Pandas Profiling after Data Preprocessing

Dataset statistics

Number of variables	12
Number of observations	99003
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	9.1 MiB
Average record size in memory	96.0 B

Variable types

NUM	11
CAT	1

Reproduction

Analysis started	2020-06-17 11:35:53.069932
Analysis finished	2020-06-17 11:38:31.077607
Duration	2 minutes and 38.01 seconds
Version	pandas-profiling v2.8.0
Command line	`pandas_profiling --config_file config.yaml [YOUR_FILE.csv]`
Download configuration	config.yaml

Warnings

`mobile_likes_received` is highly correlated with `likes_received`	High correlation
`likes_received` is highly correlated with `mobile_likes_received` and 1 other fields	High correlation
`www_likes_received` is highly correlated with `likes_received`	High correlation
`likes_received` is highly skewed (γ1 = 112.0745682)	Skewed
`mobile_likes_received` is highly skewed (γ1 = 107.5312999)	Skewed
`www_likes_received` is highly skewed (γ1 = 126.257317)	Skewed
`userid` has unique values	Unique
`friend_count` has 1962 (2.0%) zeros	Zeros
`friendships_initiated` has 2997 (3.0%) zeros	Zeros
`likes` has 22308 (22.5%) zeros	Zeros
`likes_received` has 24428 (24.7%) zeros	Zeros
`mobile_likes` has 35056 (35.4%) zeros	Zeros
`mobile_likes_received` has 30003 (30.3%) zeros	Zeros
`www_likes` has 60999 (61.6%) zeros	Zeros
`www_likes_received` has 36864 (37.2%) zeros	Zeros

userid
Real number (ℝ_≥0)

UNIQUE

Distinct count	99003
Unique (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	1597045.2079128916
Minimum	1000008
Maximum	2193542
Zeros	0
Zeros (%)	0.0%
Memory size	773.5 KiB

Quantile statistics

Minimum	1000008
5-th percentile	1060618.3
Q1	1298805.5
median	1596148
Q3	1895744
95-th percentile	2133357.1
Maximum	2193542
Range	1193534
Interquartile range (IQR)	596938.5

Descriptive statistics

Standard deviation	344059.1775
Coefficient of variation (CV)	0.2154348391
Kurtosis	-1.199556831
Mean	1597045.208
Median Absolute Deviation (MAD)	298438
Skewness	0.0001076605667
Sum	1.581122667e+11
Variance	1.183767176e+11

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
1159224	1	< 0.1%
1129202	1	< 0.1%
1055510	1	< 0.1%
1855227	1	< 0.1%
2110369	1	< 0.1%
1991449	1	< 0.1%
2128666	1	< 0.1%
1884335	1	< 0.1%
2082123	1	< 0.1%
1026848	1	< 0.1%
Other values (98993)	98993	> 99.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1000008	1	< 0.1%
1000013	1	< 0.1%
1000015	1	< 0.1%
1000038	1	< 0.1%
1000059	1	< 0.1%

Value	Count	Frequency (%)
2193542	1	< 0.1%
2193538	1	< 0.1%
2193522	1	< 0.1%
2193499	1	< 0.1%
2193485	1	< 0.1%

age
Real number (ℝ_≥0)

Distinct count	101
Unique (%)	0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	37.28022383160106
Minimum	13
Maximum	113
Zeros	0
Zeros (%)	0.0%
Memory size	773.5 KiB

Quantile statistics

Minimum	13
5-th percentile	15
Q1	20
median	28
Q3	50
95-th percentile	90
Maximum	113
Range	100
Interquartile range (IQR)	30

Descriptive statistics

Standard deviation	22.58974831
Coefficient of variation (CV)	0.6059445462
Kurtosis	1.561446767
Mean	37.28022383
Median Absolute Deviation (MAD)	10
Skewness	1.415260654
Sum	3690854
Variance	510.2967289

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
18	5196	5.2%
23	4404	4.4%
19	4391	4.4%
20	3769	3.8%
21	3671	3.7%
25	3641	3.7%
17	3283	3.3%
16	3086	3.1%
22	3032	3.1%
24	2827	2.9%
Other values (91)	61703	62.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
13	484	0.5%
14	1925	1.9%
15	2618	2.6%
16	3086	3.1%
17	3283	3.3%

Value	Count	Frequency (%)
113	202	0.2%
112	18	< 0.1%
111	18	< 0.1%
110	15	< 0.1%
109	9	< 0.1%

gender
Categorical

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	773.5 KiB

male	58749
female	40254

Common Values
Length

Value	Count	Frequency (%)
male	58749	59.3%
female	40254	40.7%

Length

Max length	6
Median length	4
Mean length	4.813187479
Min length	4

tenure
Real number (ℝ_≥0)

Distinct count	2426
Unique (%)	2.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	537.8848317727745
Minimum	0.0
Maximum	3139.0
Zeros	70
Zeros (%)	0.1%
Memory size	773.5 KiB

Quantile statistics

Minimum	0
5-th percentile	47
Q1	226
median	412
Q3	675
95-th percentile	1575
Maximum	3139
Range	3139
Interquartile range (IQR)	449

Descriptive statistics

Standard deviation	457.645601
Coefficient of variation (CV)	0.8508245147
Kurtosis	2.199181661
Mean	537.8848318
Median Absolute Deviation (MAD)	213
Skewness	1.535709166
Sum	53252212
Variance	209439.4961

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
300	173	0.2%
303	170	0.2%
242	164	0.2%
272	163	0.2%
257	161	0.2%
297	161	0.2%
285	160	0.2%
280	160	0.2%
284	158	0.2%
278	158	0.2%
Other values (2416)	97375	98.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	70	0.1%
1	60	0.1%
2	72	0.1%
3	79	0.1%
4	86	0.1%

Value	Count	Frequency (%)
3139	3	< 0.1%
3129	1	< 0.1%
3128	1	< 0.1%
3101	1	< 0.1%
3019	1	< 0.1%

friend_count
Real number (ℝ_≥0)

ZEROS

Distinct count	2562
Unique (%)	2.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	196.3507873498783
Minimum	0
Maximum	4923
Zeros	1962
Zeros (%)	2.0%
Memory size	773.5 KiB

Quantile statistics

Minimum	0
5-th percentile	3
Q1	31
median	82
Q3	206
95-th percentile	720
Maximum	4923
Range	4923
Interquartile range (IQR)	175

Descriptive statistics

Standard deviation	387.304229
Coefficient of variation (CV)	1.972511719
Kurtosis	50.09427289
Mean	196.3507873
Median Absolute Deviation (MAD)	64
Skewness	6.059008484
Sum	19439317
Variance	150004.5658

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	1962	2.0%
1	1816	1.8%
2	1117	1.1%
3	860	0.9%
5	789	0.8%
4	749	0.8%
10	737	0.7%
24	732	0.7%
6	720	0.7%
29	719	0.7%
Other values (2552)	88802	89.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	1962	2.0%
1	1816	1.8%
2	1117	1.1%
3	860	0.9%
4	749	0.8%

Value	Count	Frequency (%)
4923	1	< 0.1%
4917	1	< 0.1%
4863	1	< 0.1%
4845	1	< 0.1%
4844	1	< 0.1%

friendships_initiated
Real number (ℝ_≥0)

ZEROS

Distinct count	1519
Unique (%)	1.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	107.45247113723826
Minimum	0
Maximum	4144
Zeros	2997
Zeros (%)	3.0%
Memory size	773.5 KiB

Quantile statistics

Minimum	0
5-th percentile	1
Q1	17
median	46
Q3	117
95-th percentile	418
Maximum	4144
Range	4144
Interquartile range (IQR)	100

Descriptive statistics

Standard deviation	188.786951
Coefficient of variation (CV)	1.756934475
Kurtosis	42.53560096
Mean	107.4524711
Median Absolute Deviation (MAD)	36
Skewness	5.150757415
Sum	10638117
Variance	35640.51287

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	2997	3.0%
1	2212	2.2%
2	1551	1.6%
3	1355	1.4%
4	1352	1.4%
6	1328	1.3%
5	1328	1.3%
11	1319	1.3%
8	1314	1.3%
13	1279	1.3%
Other values (1509)	82968	83.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	2997	3.0%
1	2212	2.2%
2	1551	1.6%
3	1355	1.4%
4	1352	1.4%

Value	Count	Frequency (%)
4144	1	< 0.1%
3654	1	< 0.1%
3594	1	< 0.1%
3538	1	< 0.1%
3415	1	< 0.1%

likes
Real number (ℝ_≥0)

ZEROS

Distinct count	2924
Unique (%)	3.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	156.07878549134875
Minimum	0
Maximum	25111
Zeros	22308
Zeros (%)	22.5%
Memory size	773.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	1
median	11
Q3	81
95-th percentile	726
Maximum	25111
Range	25111
Interquartile range (IQR)	80

Descriptive statistics

Standard deviation	572.2806808
Coefficient of variation (CV)	3.666614134
Kurtosis	200.4456878
Mean	156.0787855
Median Absolute Deviation (MAD)	11
Skewness	11.02370356
Sum	15452268
Variance	327505.1777

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	22308	22.5%
1	6928	7.0%
2	4434	4.5%
3	3240	3.3%
4	2507	2.5%
5	2027	2.0%
6	1806	1.8%
7	1618	1.6%
8	1430	1.4%
9	1381	1.4%
Other values (2914)	51324	51.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	22308	22.5%
1	6928	7.0%
2	4434	4.5%
3	3240	3.3%
4	2507	2.5%

Value	Count	Frequency (%)
25111	1	< 0.1%
21652	1	< 0.1%
16732	1	< 0.1%
16583	1	< 0.1%
14799	1	< 0.1%

likes_received
Real number (ℝ_≥0)

HIGH CORRELATION
SKEWED
ZEROS

Distinct count	2681
Unique (%)	2.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	142.6893629485975
Minimum	0
Maximum	261197
Zeros	24428
Zeros (%)	24.7%
Memory size	773.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	1
median	8
Q3	59
95-th percentile	561
Maximum	261197
Range	261197
Interquartile range (IQR)	58

Descriptive statistics

Standard deviation	1387.919613
Coefficient of variation (CV)	9.726861091
Kurtosis	17384.94
Mean	142.6893629
Median Absolute Deviation (MAD)	8
Skewness	112.0745682
Sum	14126675
Variance	1926320.851

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	24428	24.7%
1	7305	7.4%
2	4541	4.6%
3	3347	3.4%
4	2669	2.7%
5	2373	2.4%
6	1873	1.9%
7	1680	1.7%
8	1538	1.6%
9	1351	1.4%
Other values (2671)	47898	48.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	24428	24.7%
1	7305	7.4%
2	4541	4.6%
3	3347	3.4%
4	2669	2.7%

Value	Count	Frequency (%)
261197	1	< 0.1%
178166	1	< 0.1%
152014	1	< 0.1%
106025	1	< 0.1%
82623	1	< 0.1%

mobile_likes
Real number (ℝ_≥0)

ZEROS

Distinct count	2396
Unique (%)	2.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	106.11629950607558
Minimum	0
Maximum	25111
Zeros	35056
Zeros (%)	35.4%
Memory size	773.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	4
Q3	46
95-th percentile	481.9
Maximum	25111
Range	25111
Interquartile range (IQR)	46

Descriptive statistics

Standard deviation	445.2529851
Coefficient of variation (CV)	4.195896268
Kurtosis	360.9885806
Mean	106.1162995
Median Absolute Deviation (MAD)	4
Skewness	14.16123656
Sum	10505832
Variance	198250.2207

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	35056	35.4%
1	6297	6.4%
2	3941	4.0%
3	2917	2.9%
4	2265	2.3%
5	1794	1.8%
6	1598	1.6%
7	1395	1.4%
8	1212	1.2%
9	1149	1.2%
Other values (2386)	41379	41.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	35056	35.4%
1	6297	6.4%
2	3941	4.0%
3	2917	2.9%
4	2265	2.3%

Value	Count	Frequency (%)
25111	1	< 0.1%
21652	1	< 0.1%
16732	1	< 0.1%
14039	1	< 0.1%
13529	1	< 0.1%

mobile_likes_received
Real number (ℝ_≥0)

HIGH CORRELATION
SKEWED
ZEROS

Distinct count	2004
Unique (%)	2.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	84.1204912982435
Minimum	0
Maximum	138561
Zeros	30003
Zeros (%)	30.3%
Memory size	773.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	4
Q3	33
95-th percentile	317
Maximum	138561
Range	138561
Interquartile range (IQR)	33

Descriptive statistics

Standard deviation	839.8894437
Coefficient of variation (CV)	9.984362083
Kurtosis	15522.64932
Mean	84.1204913
Median Absolute Deviation (MAD)	4
Skewness	107.5312999
Sum	8328181
Variance	705414.2777

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	30003	30.3%
1	8243	8.3%
2	4948	5.0%
3	3608	3.6%
4	2944	3.0%
5	2383	2.4%
6	2022	2.0%
7	1745	1.8%
8	1521	1.5%
9	1437	1.5%
Other values (1994)	40149	40.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	30003	30.3%
1	8243	8.3%
2	4948	5.0%
3	3608	3.6%
4	2944	3.0%

Value	Count	Frequency (%)
138561	1	< 0.1%
131244	1	< 0.1%
89911	1	< 0.1%
73333	1	< 0.1%
43410	1	< 0.1%

www_likes
Real number (ℝ_≥0)

ZEROS

Distinct count	1726
Unique (%)	1.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	49.96242538104906
Minimum	0
Maximum	14865
Zeros	60999
Zeros (%)	61.6%
Memory size	773.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	7
95-th percentile	208
Maximum	14865
Range	14865
Interquartile range (IQR)	7

Descriptive statistics

Standard deviation	285.5601519
Coefficient of variation (CV)	5.715498191
Kurtosis	449.1484832
Mean	49.96242538
Median Absolute Deviation (MAD)	0
Skewness	16.91102529
Sum	4946430
Variance	81544.60033

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	60999	61.6%
1	4697	4.7%
2	2760	2.8%
3	1948	2.0%
4	1419	1.4%
5	1202	1.2%
6	1081	1.1%
7	897	0.9%
8	792	0.8%
9	757	0.8%
Other values (1716)	22451	22.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	60999	61.6%
1	4697	4.7%
2	2760	2.8%
3	1948	2.0%
4	1419	1.4%

Value	Count	Frequency (%)
14865	1	< 0.1%
12903	1	< 0.1%
11077	1	< 0.1%
10763	1	< 0.1%
10627	1	< 0.1%

www_likes_received
Real number (ℝ_≥0)

HIGH CORRELATION
SKEWED
ZEROS

Distinct count	1636
Unique (%)	1.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	58.56883124753795
Minimum	0
Maximum	129953
Zeros	36864
Zeros (%)	37.2%
Memory size	773.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	2
Q3	20
95-th percentile	227
Maximum	129953
Range	129953
Interquartile range (IQR)	20

Descriptive statistics

Standard deviation	601.416348
Coefficient of variation (CV)	10.26853934
Kurtosis	23812.2491
Mean	58.56883125
Median Absolute Deviation (MAD)	2
Skewness	126.257317
Sum	5798490
Variance	361701.6237

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	36864	37.2%
1	8513	8.6%
2	5111	5.2%
3	3586	3.6%
4	2828	2.9%
5	2317	2.3%
6	1918	1.9%
7	1602	1.6%
8	1445	1.5%
9	1373	1.4%
Other values (1626)	33446	33.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	36864	37.2%
1	8513	8.6%
2	5111	5.2%
3	3586	3.6%
4	2828	2.9%

Value	Count	Frequency (%)
129953	1	< 0.1%
62103	1	< 0.1%
39605	1	< 0.1%
39213	1	< 0.1%
34039	1	< 0.1%

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Count
Matrix

First rows

	userid	age	gender	tenure
0	2094382	14	male	266.0
1	1192601	14	female	6.0
2	2083884	14	male	13.0
3	1203168	14	female	93.0
4	1733186	14	male	82.0
5	1524765	14	male	15.0
6	1136133	13	male	12.0
7	1680361	13	female	0.0
8	1365174	13	male	81.0
9	1712567	13	male	171.0

Last rows

	userid	age	gender	tenure	friend_count	friendships_initiated	likes	likes_received	mobile_likes	mobile_likes_received	www_likes	www_likes_received
98993	1654565	19	male	394.0	4538	4144	4501	15088	4435	5961	66	9127
98994	2063006	20	female	402.0	1988	332	7351	106025	7248	73333	103	32692
98995	1132164	20	female	699.0	3611	973	4507	7768	4414	6909	93	859
98996	1668695	24	female	182.0	2938	1272	6018	17765	5843	11708	175	6057
98997	1458985	28	female	290.0	2218	1618	4626	10268	4290	4250	336	6018
98998	1268299	68	female	541.0	2118	341	3996	18089	3505	11887	491	6202
98999	1256153	18	female	21.0	1968	1720	4401	13412	4399	10592	2	2820
99000	1195943	15	female	111.0	2002	1524	11959	12554	11959	11462	0	1092
99001	1468023	23	female	416.0	2560	185	4506	6516	4506	5760	0	756
99002	1397896	39	female	397.0	2049	768	9410	12443	9410	9530	0	2913

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows